Ranging techniques exist which utilize triangulation or stereoscopic viewing of an object to ascertain the range of the object. It will appreciated that in stereoscopic or triangulation systems, the object is viewed from two distinct positions from which data the range to various points on the object can be ascertained. Because of the observation of the object from two distinct locations, the consequent stereopsis is plagued with several problems.
The first problem is the so-called "missing parts" Problem in which all parts of the scene are not visible at both viewing locations. This results in "holes" in the range image due to the occlusion or shadowing effect in which parts of the scene are in front of and occlude other parts of the scene. When viewed from different angles required by triangulation or stereo techniques, there are thus various parts of the scene for which no data can be obtained.
Additionally when periodic structured light patterns, e.g. lines or grids, are used, the system which is detecting the range of various localized portions of the scene can be confused as to which parts of the pattern the camera is looking at. This confusion leads to erroneous range measurements. Note the uncertainty is due to the spatial separation between the points at which the object is viewed.
Triangulation type techniques have been described by the following investigators in the indicated publications: R. A. Jarvis, "A Perspective on Range Finding Techniques for Computer Vision," IEEE Trans. on Patt. Anal. and Mach. Intell., vol PAMI-5, no. 2, 122-139, March 1983; K. S. Fu, R. C. Gonzales, C. S. G. Lee, "Robotics: Control, Sensing, Vision, and Intelligence," McGraw-Hill, 1987; K. Araki, Y. Sato, S. Parthasarathy, "High Speed Range Finder," SPIE vol. 850, Optics, Illumination, and Image Sensing for Machine Vision II, 184-188, Cambridge, Nov. 1987; K. L. Boyer, A. C. Kak, "Color-Encoded Structured Light for Rapid Active Sensing," IEEE Trans. on Patt. Anal. and Mach. Intell., vol. PAMI-9, no. 1, 14-28, January 1987; W. Frobin, E. Hierholzer, "Rasterstereography: A Photogrammetric Method for Measurement of Body Surfaces," Photogrammetric Engineering and Remote Sensing, vol. 47, no. 12., 1717-1724, Dec. 1981.
In order to solve the above-mentioned problems with stereoscopic or triangulation techniques, obtaining range from the defocus of an object imaged on a camera image plane has been suggested by A. P. Pentland in an article entitled, "A New Sense for Depth of Field," IEEE Trans. on Patt. Anal. and Mach. Intell., vol. PAMI-9, no. 4, 523-531, July 1987.
In this paper a passive system is provided in which an object is viewed by a camera having a wide aperture lens and a narrow aperture lens, with the extent of blur of the object providing for localized measurements of range from the camera to the object. The Pentland system however requires that the object contains high spatial frequency components across the scene in order to be able to measure local blur for each visible point of the scene. What will be apparent from the Pentland system is that the utilization of a passive technique cannot provide for range measurements to objects such as plane walls which do not naturally have high spatial frequency components.
Note, since cameras having wide and narrow aperture lenses are used in the Pentland system, two images are produced that are identical except for depth of field, because change of aperture does not affect the position of the image features. Since differing aperture size causes differing focal gradients, the same point will be focused differently in the two images. Note that the magnitude of this difference is a simple function of the distance between the viewer and imaged point. Here the small depth of field, large aperture optics provides a sharp image baseline from which to measure the blur detected by the large depth of field, small aperture optics.
In summary, in the Pentland system, to obtain an estimate of range, all that is necessary is to compare blur at corresponding points of the two images and measure the change in focus. Because the two images are identical except for aperture size, they may be compared directly. This is because there is no matching problem as there is with stereo or motion algorithms. It is thus possible to recover absolute distance by a comparison of the local blur of the two images.
It will be noted also that none of the range cameras mentioned above can pick up disparities in flat regions of an image, with both the stereo systems and the Pentland system suffering from image dependency, absent the use of structured light. Structured light, as used herein, refers to the projection of a pattern onto an object in which the pattern is one containing high spatial frequency components.
Note other publications relating to range cameras include: M. Subbarao, "Parallel Depth Recovery by Changing Camera Parameters," IEEE 2nd Intern. Conf. on Computer Vision, 149-155, Dec. 1988; T. Darell, K. Wohn, "Pyramid Based Depth from Focus," Proc. IEEE Conf. on Computer Vision and Pattern Recognition '88, 504-509, June 1988; V. M. Bove, Jr., "Synthetic Movies Derived from Multi-Dimensional Image Sensors," Ph.D. thesis, Media Arts and Sciences Section, Mass. Inst. of Techn., 1989.
There are numerous applications for which a range camera can be put. With not only reflected luminescence as a function of position but also range as a function of position, a range camera can be utilized as a major aid in post-production processing in which originally photographed scenes can be altered and/or manipulated both as to spatial orientation and lighting. Range cameras give a new dimension to so-called "blue boarding" in which certain images are made to appear ahead of or behind other images. The ability to obtain three dimensional data through the utilization of a range camera aides in robot vision, image processing, image understanding, graphics, animation, three dimensional television or motion pictures, three dimensional portraits, human face recognition for security purposes, anthropometry, medical applications, visual inspection of parts including optical comparisons, modelization, vehicle guidance, mold fabrication CAD/CAM, automatic assembly, object reproduction including mold reproduction, and certain museum applications. The derivation of three dimensional images or information to produce these images is useful in a variety of research activities, for example integration of multiple views of the same object to form a model; creation of a library of object models to investigate searching algorithms and image description; and development of interactive procedures to measure, manipulate, and edit three dimensional information.
For these and other applications any range camera should have the following properties: The system should provide a sensing process which is relatively fast. The range camera should be able to run at the same rates as an ordinary video camera, at least at 30 frames per second. Moreover the sensing process should be active in the sense of using structured light. Passive viewing of scenes or objects does not achieve the required range content of the entire scene, should the object not have the type of discontinuities or high spatial frequencies required. Thus the system should be capable of achieving a reliable range signal independent of surface texture and illumination. Additionally, laser illumination should be avoided as a light source. In applications which involve range images of people, even if there is no health hazard, there is a considerable psychological barrier to using lasers. It will of course be appreciated that spatial resolution as well as depth resolution of the range image should be as high as possible. Finally the technique should be inexpensive and robust.